Grid: the empty-digest quorum guard, and the instrument on the manifest - #33
Merged
Conversation
A digest with no comparable call agrees with every other such digest — every field is absent on both sides. `a_shapeless_digest_agrees_with_nothing_real` already asserted exactly that and called it "equally uninformative", and I did not follow the thought through: two nodes whose analysis failed would agree, canonicalize a unit with no content, and be credited for it. `digest::has_content` now answers whether a digest is about anything, and the validator refuses to canonicalize when the winning cluster is empty. One discrete call is enough. The Navigator side is being fixed in the same pass — a node now fails its unit instead of submitting an empty digest. This guard is here anyway, because a node is untrusted by construction and that is the entire premise adaptive replication rests on. A check that only runs on the honest side is not a check. Found by a review of the Navigator branch, which traced the consequence back across the repository boundary. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SequryftjcFkLnjQRMKCx3
The node cannot pick a mapper preset without it. A read set with no mate is either a long read or a single-end short read, and those need different presets — a long read mapped under a short-read preset does not fail, it produces alignments that look right and are wrong. `crawl_project` already stores ENA's instrument model on the library, so this only publishes a value we hold. The Navigator side then feeds it to `Preset::infer`, which errors rather than guesses on an instrument it does not recognise. Found by the third review of the Navigator branch: the node was choosing its preset from the mate count, so every single-end Illumina run would have been mapped as HiFi. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SequryftjcFkLnjQRMKCx3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two AppView-side changes that came out of reviewing the Navigator edge
(decodingus-navigator#61).
Both are guards the AppView needs regardless of how well-behaved a node is.
Two empty digests must not form a quorum
A digest with no comparable call agrees with every other such digest — every field is
absent on both sides.
a_shapeless_digest_agrees_with_nothing_realalready assertedexactly that and called it "equally uninformative", and I did not follow the thought
through: two nodes whose analysis failed would agree, canonicalize a unit with no
content, and be credited for it.
digest::has_contentnow answers whether a digest is about anything, andgrid-validaterefuses to canonicalize an empty winning cluster. One discrete callis enough.
The Navigator side was fixed in the same pass — a node now fails its unit rather than
submitting an empty digest. This guard is here anyway, because a node is untrusted
by construction and that is the whole premise adaptive replication rests on. A check
that only runs on the honest side is not a check.
The instrument on the curation manifest
A node cannot choose a mapper preset without it. A read set with no mate is either a
long read or a single-end short read, and those need different presets — a long read
mapped under a short-read preset does not fail, it produces alignments that look
right and are wrong.
crawl_projectalready stores ENA's instrument model on the library, so thispublishes a value we already hold. Navigator feeds it to
Preset::infer, which errorsrather than guesses on an unrecognised instrument, so the unit goes back to the pool
instead of a result of unknown quality reaching the quorum.
Without this, every single-end Illumina run in the catalogue would have been mapped as
HiFi.
Verifying
du-db's grid tests are live-Postgres and skip withoutDATABASE_URL. To run them:Serially, with ≥4 GB — each test migrates its own database.
CI
origin/mainstill failscargo clippy --workspace -- -D warningsunder currentstable clippy, in files this branch does not touch (
du-db/src/place.rs:297is_none_orvs the declared MSRV 1.80, plusdu-web/src/origins_layout.rs:505,512).Raised on #32 and still unaddressed; happy to fix in a separate PR if you want it.
🤖 Generated with Claude Code
https://claude.ai/code/session_01SequryftjcFkLnjQRMKCx3